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Chapter  1  Overview  of  the  Report 


A  Collaboration  in 
Implementing  Team  Risk 
Management 


Abstract  This  report  presents  results  of  a  collaborative  development  effort  to  transi¬ 

tion  the  Software  Engineering  Institute  (SEI)  team  risk  management  pro¬ 
cess  into  practice.  The  collaboration  involved  a  DoD  program  office  (cus¬ 
tomer),  a  commercial  contractor  (supplier),  and  the  SEI  in  an  effort  that  in¬ 
cluded  development,  test,  and  refinement  of  the  team  risk  management 
approach.  The  focus  of  the  report  is  on  the  results  of  the  collaboration  be¬ 
tween  the  contractor  organization  and  the  SEI. 


Chapter  1  Overview  of  the  Report 


Collaboration 

Effort 

In  the  second  quarter  of  1993,  the  Software  Engineering  Institute  (SEI)  en¬ 
tered  into  a  collaborative  agreement  with  a  Department  of  Defense  (DoD) 
client  to  transition  the  team  risk  management  process  into  a  software  devel¬ 
opment  project  —  the  pilot  project. 

Objective 

This  report  focuses  on  the  details  and  presents  the  results  of  the  collabora¬ 
tion  with  the  contractor  (supplier)  organization. 

Pilot  Project 

The  transition  effort  involved  a  pilot  implementation  of  team  risk  manage¬ 
ment  within  a  software-intensive  airborne  system  development  program. 

Lessons 

Learned 

By  using  the  collaborative  events  conducted  between  the  SEI  and  the  con¬ 
tractor  organization  as  a  framework,  this  report  focuses  on  the  lessons 
learned  in  the  effort  to  transition  team  risk  management  into  the  pilot  pro¬ 
gram. 
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Summary  of 
the  Report 


Confidential 


Bracketed 

Phrases 


An  overview  of  the  team  risk  management  approach  is  presented  and  the 
process  of  establishing  the  foundations  for  the  transition  is  discussed.  The 
major  process  steps  in  the  collaboration  are  then  reviewed  and  the  relevant 
issues,  problems,  successes,  and  lessons  learned  are  presented.  The  paper 
concludes  with  a  review  of  the  collaboration  and  a  summary  of  the  lessons 
learned. 


Because  of  confidentiality  agreements  established  between  the  SEI  and  cli¬ 
ent  organizations,  the  specific  pilot  program  is  not  identified  in  this  report. 


Throughout  this  report,  brackets  []  are  used  to  identify  editorial  additions 
that  were  employed  to  improve  the  clarity  of  quotes  and  comments. 
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Chapter  2  Team  Risk  Management 


Overview  of  Team  risk  management  [Higuera  93],  [Dorofee  93],  [Higuera  95]  enables 
Team  Risk  the  customer  and  supplier  to  work  together  cooperatively,  continuously 

Management  managing  risks  throughout  the  life  cycle  of  a  software-dependent  develop¬ 

ment  program.  It  is  built  on  the  principles  of  risk  management  and  a  philos¬ 
ophy  of  cooperative  teams. 


Cooperative  The  team  risk  management  approach  establishes  a  cooperative  working  en- 
Working  vironment  throughout  all  levels  of  a  program  that  gives  everyone  in  the  pro- 

Environment  gram  the  ability  and  motivation  to  look  ahead  and  to  handle  risks  before  they 

become  problems. 


Team  Risk  The  model  for  team  risk  management  is  shown  in  Figure  1.  Each  function 
Management  has  a  set  of  activities  backed  by  processes,  methods,  and  tools  that  encour- 
Model  age  and  enhance  communication  and  teamwork. 


Figure  1:  Team  Risk  Management  Model 
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Management  Table  1  summaries  each  of  the  team  risk  management  functions.  Communi- 
Functions  cation  is  an  integral  part  of  all  these  activities.  More  details  can  be  found  in 
the  Continuous  Risk  Management  Guidebook} 


Table  1: 

Team  Risk  Management  Functions 

Function 

Description 

•  Initiate 

Recognize  the  need  and  commit  to  create  the 
team  culture.  Either  customer  or  supplier  may 
initiate  team  activity,  but  both  must  commit  to 
sustain  the  teams. 

•  Team 

^ _ K 

Formalize  the  customer  and  supplier  team,  and 
merge  the  viewpoints  to  form  a  shared  product 
vision.  Systematic  methods  that  are  applied 
periodically  and  jointly  establish  a  shared 
understanding  of  the  project  risks  and  their 
relative  importance.  Establish  joint  information 
base  of  risks,  priorities,  metrics,  and  action  plans. 

•  Identify 

Search  for  and  locate  risks  before  they  become 
problems.  Identify  risks  and  set  program 
priorities  to  arrive  at  a  joint  understanding  of  what 
is  important. 

Identify  new  risks  and  changes. 

•  Analyze 

^ 

Process  risk  data  into  decision-making 
information  to  determine  what  is  important  to  the 
project,  to  set  priorities,  and  to  allocate  resources. 

Group  risks  and  quantify  impact,  probability,  and 
timeframe. 

1.  Continuous  Risk  Management  Guidebook.  Dorofee,  Audrey;  Walker,  Julie;  Alberts,  Christopher; 
Higuera,  Ronald;  Murphy,  Richard  L.  &  Williams,  Ray  C.  (to  be  published  in  June  1996).  Pittsburgh, 
PA.  Software  Engineering  Institute,  Carnegie  Mellon  University. 


4 


CMU/SEI-95-TR-016 


Chapter  2  Team  Risk  Management 


Table  1:  Team  Risk  Management  Functions  (Continued) 


Function 

Description 

•  Plan 

^ 

Translate  risk  information  into  decisions  and 
mitigating  actions  (both  present  and  future),  and 
implement  those  actions.  Joint  risks  require  a 
team  process  to  develop  mitigation  plans. 

Establish  the  mitigation  plans  for  the  risks. 

♦  Track 

fiound  ■ - - 

Monitor  risk  indicators  and  mitigation  plans. 
Indicators  and  trends  provide  information  to 
activate  plans  and  contingencies.  These  are  also 
reviewed  periodically  to  measure  progress  and 
identify  new  risks. 

Maintain  visibility  of  risks,  project  priority,  and 
mitigation  plans. 

•  Control 

CWT  PW^T*^*"'*'***'  I 

»u>ni  ^  ^  \ 

Correct  for  deviations  from  the  risk  mitigation 
plans.  Actions  can  lead  to  corrections  in  products 
or  processes.  Any  action  may  lead  to  joint 
resolution.  Changes  to  risks,  risks  that  become 
problems,  or  faulty  plans  require  adjustments  in 
plans  or  actions. 

Maintain  the  level  of  risk  acceptable  to  the  project 
managers. 

•  Communicate 

Provide  information  and  feedback  internal  and 
external  to  the  project  on  the  risk  activities, 
current  risks,  and  emerging  risks. 

Communication  occurs  formally  as  well  as 
informally. 

Establish  continuous,  open  communication. 

Formal  communication  about  risks  and  action 
plans  is  integrated  into  existing  technical 
interchange  meetings,  design  reviews,  and  user 
requirements  meetings. 
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Chapter  3  Collaboration  Description 


Overview  The  collaborative  transition  effort  spanned  more  than  1 8  months.  The  effort 

involved  personnel  from  three  organizations:  the  DoD  program  office  (cus¬ 
tomer),  contractor  (supplier),  and  the  SEI.^ 


Aspects  of  the  The  collaboration  addressed  three  key  issues  relating  to  the  installation  of 
Collaboration  team  risk  management: 

1 .  transition  of  the  team  risk  management  approach 

2.  facilitation  of  the  team  risk  management  processes 

3.  development  and  enhancement  of  the  methods. 


Transition  The  transition  of  team  risk  management  into  routine  practice  within  the  con¬ 
tractor  organization  and  the  program  office  was  a  primary  goal. 


Facilitation  The  SEI  was  directly  involved  in  the  facilitation  of  the  team  risk  manage¬ 
ment  processes  within  the  pilot  program  and  was  an  integral  part  of  the  tran¬ 
sition  and  development  processes. 


Development 

and 

Enhancement 


At  the  start  of  the  collaboration,  the  processes  and  most  of  the  methods  and 
tools  of  team  risk  management  had  been  developed,  but  many  aspects  of  the 
approach  had  not  been  extensively  tested.  Much  of  the  collaboration  ad¬ 
dressed  extending  and  customizing  the  team  risk  management  approach 
through  a  cooperative  effort  between  the  contractor  and  the  SEI. 


1.  Throughout  this  report,  the  customer  and  supplier  of  the  team  risk  management  model  are  the  govern¬ 
ment  and  contractor,  respectively. 
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Section  1  Collaboration  Goals  and  Background 


Goals  of  the  The  broad  goals  of  the  collaboration  were  to 

Collaboration  ] .  transition  the  team  risk  management  process  into  routine  practice  within 
the  pilot  program 

2.  lay  the  foundations  for  broadening  the  processes  and  methods  into  other 
programs  within  both  the  government  and  contractor  organizations 

3.  modify,  enhance,  and  expand  the  team  risk  management  methods  to 
meet  specific  client  needs  and  to  improve  the  quality  and  effectiveness 
of  the  approach. 

Establishing  a  The  pilot  program  and  the  corporate  organization  were  selected  based  upon 

Working  the  following  characteristics: 

Relationship  •  positive  attitude  toward  change  and  improvement 
•  commitment  to  software  process  improvement 


Positive  In  general,  the  program  was  characterized  by  a  positive  attitude  toward 

Attitude  improvement  and  a  receptiveness  to  new  approaches.  Both  the  government 

Toward  program  office  and  the  contractor  had  a  strong,  progressive  management 

Change  and  team  assigned  to  the  program.  In  fact,  risks  had  been  addressed  in  the  initial 
Improvement  phases  of  the  program,  and  the  contractor  was  addressing  risk  management 
indirectly  as  part  of  problem  and  issue  reporting  and  tracking. 


Commitment  Within  the  program  office  and  the  contractor  organization,  there  was  a  com- 
to  Software  mitment  to  and  an  active  involvement  in  software  process  improvement. 
Process 
Improvement 
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Section  2  Structure  and  Roles 


Structure  of  The  collaboration  involved  personnel  from  the  government,  contractor,  and 
the  the  SET  Personnel  from  these  organization  were  involved  in  two  distinct 

Collaboration  sets  of  activities: 

•  conducting  team  risk  management  processes  or 

•  facilitating  team  risk  management  and  transition  processes 

Three  distinct  teams  were  formed  for  the  collaboration: 

•  government-contractor  team 

•  government  transition  team 

•  contractor  transition  team 

Composition  The  organizations  and  personnel  that  formed  the  teams  for  the  collaboration 
of  the  Teams  are  shown  in  Figure  2. 


Teams 
Involved  in 
the 

Collaboration 


Figure  2:  Team  Composition  Model 
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Government  -  The  government-contractor  team  consisted  of  two  program  managers  (PM), 

Contractor  government  and  contractor,  and  at  least  two  key  program  personnel  from 

Team  each  organization.  This  team  met  periodically  to  conduct  the  joint  activities 

of  team  risk  management,  e.g.,  team  reviews  (refer  to  Chapter  4,  Section  12) 
and  joint  action  planning  (refer  to  Chapter  4,  Section  13). 


Government  Two  individuals  were  assigned  to  the  collaboration  from  the  systems  engi- 

Transition  neering  [S/E]  staff  of  the  government  program  offices.  These  individuals. 

Team  together  with  two  SEI  staff,  formed  the  government  transition  team. 

It  was  planned  that  these  individuals  would  participate  throughout  the  col¬ 
laboration,  but  due  to  government  organizational  changes  the  composition 
of  this  team  was  not  stable.  In  general,  most  of  the  responsibilities  of  this 
team  were  handled  by  the  SEI  team  members. 


Contractor  Two  individuals  from  the  contractor’s  software  engineering  process  group 
Transition  (SEPG)  organization  were  assigned  to  the  collaboration.  These  individuals 

Team  participated  throughout  the  full  duration  of  the  effort,  and  together  with  two 

SEI  staff  formed  the  contractor  transition  team. 


SEI  There  was  a  total  of  four  members  of  the  technical  staff  from  the  SEI.  Two 

Participation  as  members  of  the  government  transition  team  and  two  others  as  members 

of  the  contractor  transition  team. 


Roles  Transition  team  members  from  the  government  and  contractor  organiza¬ 

tions  were  trained  in  the  team  risk  management  approach,  participated  in  the 
planning,  contributed  to  the  development  effort,  and  facilitated  the  team  risk 
management  processes  within  their  organizations  throughout  the  duration  of 
the  collaboration. 


The  role  of  these  key  individuals  was  primarily  as  transition  agents  [Fowler 
92]  for  the  process,  but  they  also  served  as  codevelopers  in  the  design, 
development,  and  evolution  of  the  methods. 

Since  one  of  the  objectives  of  the  collaboration  was  to  transition  the 
technology  into  the  entire  organization,  the  collaboration  efforts  also 
included  considerations  of  organization-wide  needs  and  the  issues  relating 
to  broad-based  institutionalization  of  the  practices.  Their  insight  into  unique 
organizational  issues  and  their  knowledge  of  the  organization’s  processes 
facilitated  the  transition. 


Contribution 
of  the 
Transition 
Team 
Members 
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Participation  Program  personnel  throughout  both  organizations  participated  in  the  activ- 
of  Other  ities  of  team  risk  management.  This  participation  involved  risk  identifica- 

Personnel  tion,  risk  management,  and  support  of  the  risk  mitigation  activities. 
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Chapter  4  Implementation 


Team  Risk  The  efforts  reported  here  were  used,  in  part,  in  the  development  of  the  team 
Management  risk  management  implementation  process  roadmap  shown  in  Figure  3  [Dor- 
Roadmap  ofee  95].  The  actual  implementation  of  the  collaborative  effort  closely  fol¬ 

lowed  the  sequence  of  the  roadmap. 


Applying  Team 
Risk  Management 
-  A  Roadmap 


[CUSTOMER'S  ORGANIZATION 


Adapt 

TRM 

to 

Project 


Project 

Risk 

Baseline 


Establish 

Risk 

Tracking 


Plan 

First 

Risks 


Establish 

Continuous 

Processes 


Repeat  & 
improve 
Continuous 
Processes 


Conduct 

Initial 

Training 


Start 

RRIA 


Build 

Customer- 

Supplier 

Team 


Joint  Action 
Planning 


Team 

Review 


[SUPPUER'S  ORGANIZATION 


EaUMI^ 

ContitMJOUs 

ProcMAu 


InstaiJ 


Continue 


*  Routine  Risk  Identification  and  Analysis 


Figure  3:  Team  Risk  Management  Roadmap 


Process 

Roadmap 


There  are  three  major  phases  for  the  implementation  of  team  risk  manage¬ 
ment:  start,  install,  and  continue.  These  phases  and  the  steps  involved  with¬ 
in  each  phase  are  shown  in  the  roadmap  graphic. 
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Customer 

and 

Contractor 

Timelines 


The  customer  (government)  and  supplier  (contractor)  timelines  are  depicted 
separately  in  the  roadmap  as  broad  arrows  that  run  from  left  to  right  across 
the  roadmap.  The  implementation  begins  with  the  sponsor’s  commitment  to 
team  risk  management. 

Note:  The  steps  are  shown  grayed  out  on  the  supplier  timeline,  indicating 
that  these  activities  are  identical  to  those  along  the  customer  timeline. 


Joint 

Activities 

Timeline 


The  joint  activities  timeline,  located  at  the  center  of  the  roadmap,  parallels 
the  separate  customer  and  supplier  timeline.  The  steps  identified  on  this 
timeline  are  the  joint  team  risk  management  activities  conducted  by  the  cus¬ 
tomer-supplier  team. 


Events  The  major  collaboration  events,  the  steps  in  the  roadmap  that  each  supports. 

Summary  and  the  section  of  the  document  discussing  the  event  are  shown  in  Table  2. 


Table  2:  Major  Collaboration  Events 


Collaboration 

Event 

Description 

Section 

Meetings  to  select 
program  for 
collaboration 

A  series  of  meetings  among  the  DoD  program 
office  personnel  and  SEI  personnel  to  discuss 
expectations  and  candidate  programs  for  the 
collaboration 

Section  1: 
Establish 
Sponsorship 

Introductory  meeting 
with  contractor 

A  formal  meeting  that  introduced  the  team  risk 
management  approach  and  the  details  of  the 
collaboration  to  contractor  personnel.  The 
contractor  took  advantage  of  this  meeting  to 
include  personnel  from  other  groups  within  the 
larger  organization,  those  outside  the  pilot 
program. 

Section  Iz 

Introductory 

Presentations 

Training 

Formal  training  of  transition  team  members 
from  the  supplier  (contractor) 

Section  3: 

Initial  Training 

General  presentation 

A  general  presentation  of  the  baseline 
activities  and  the  continuous  team  risk 
management  processes  for  all  members  of  the 
pilot  project  who  were  scheduled  to  participate 
in  the  pilot  implementation 

Section  5: 

Project  Risk 
Baseline 

Interviews  -Baseline 

The  group  interviews  and  meetings  that 
constitute  the  baseline.  These  were  conducted 
to  identify  and  analyze  the  baseline  set  of  risks. 

Section  5: 

Project  Risk 
Baseline 
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Table  2:  Major  Collaboration  Events  (Continued) 


Collaboration 

Event 

Description 

Section 

Initial  Planning 
Meeting 

A  meeting  to  support  the  planning  of  the 
baseline  set  of  risks.  This  consisted  of 
discussing,  modifying  as  needed,  and 
finalizing  the  planning  methods  and  tools  for 
use  in  the  initial  phases  of  the  collaboration. 

Section  9: 

Plan  First  Risks 

Team  Review 

Regular  meetings  held  between  the 
government  and  contractor  to  review,  discuss, 
and  reprioritize  program-wide  risks.  These  are 
key  events  within  the  team  risk  management 
approach. 

Section  10: 

Build 

Government 
and  Contractor 
Team  and  First 
Team  Review 

Planning  Meeting 
(contractor) 

A  formal  planning  meeting  of  the  contractor 
personnel  conducted  by  the  SEI.  This  was  the 
first  use  of  the  group  planning  methods  on  the 
program  and  addressed  one  of  the  highest 
priority  risks  faced  by  the  program. 

Section  11: 

Joint  Action 
Planning 

Joint  Planning  Session 

A  group  planning  meeting  facilitated  by  the 
SEI,  and  involving  both  the  government  and 
contractor  personnel.  Key  risks  requiring  input 
and  involvement  of  both  partners  (government 
and  contractor)  to  effectively  resolve  were 
addressed  in  these  sessions. 

Section  11: 

Joint  Action 
Planning 

Coordination 

Meetings 

Regular  coordination  meetings  held  with  the 
SEI  and  contractor  personnel.  These  would 
generally  span  a  full  day  and  involve  program 
technical  and  management  personnel,  as  well 
as  the  members  of  the  transition  team  from  the 
contractor  organization. 

Section  12: 
Establish 
Continuous 
Processes 

Closure  session 

A  final  group  meeting  between  the  SEI  and 
contractor  personnel  to  formally  close  the 
collaboration  and  to  finalize  a  set  of  lessons 
learned  and  areas  for  improvement  of  the 
collaboration  and  team  risk  management 
processes 

Section  14: 
Closure  of  the 
Collaboration 

Transition  Follow-up 

A  project  risk  baseline  conducted  by  the 
contractor  transition  team  personnel  on 
another  project,  observed  by  the  SEI 

Section  15: 
Continuation 
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Section  1  Establish  Sponsorship 


Description  Establish  Sponsorship  is  the  first  step  in  implementing  team  risk  manage¬ 
ment,  showing  that  management  personnel 

•  believe  the  specific  risk  management  program  outlined  to  them  is  critical 
to  the  program 

•  are  willing  to  commit  suitable  resources  to  it 

•  are  committed  to  its  success 


Sponsorship 

and 

Commitment 
for  the 

Collaboration 


The  DoD  program  office  was  supporting  the  SEI  Risk  Program  and  was 
providing  the  sponsorship  for  the  collaboration.  The  initial  activities  of  the 
collaboration  involved  selecting  an  appropriate  program  (customer  and 
supplier)  for  the  collaboration  and  securing  the  commitment  of  all  parties  to 
the  effort.  Based  upon  discussions  and  reviews  involving  the  SEI  and 
program  office  staff,  a  candidate  program  was  identified  and  steps  were 
initiated  to  secure  the  commitment  of  key  management  personnel  in  both  the 
government  and  contractor  organizations. 


Initial 
Meetings  to 
Secure 
Commitment 


Visits  to  the  government  program  office  and  contractor  were  conducted 
toward  achieving  an  understanding  of  the  nature,  level  of  personnel 
required,  major  process  steps,  expected  outcome  of  the  collaboration,  and 
benefits  of  a  successful  adoption  of  team  risk  management. 

The  objective  was  to  achieve  the  commitment  of  both  the  government  and 
the  contractor  program  managers,  in  addition  to  other  key  management 
personnel. 


Program 

Office 

Commitment 


While  management  within  the  government  program  office  was  interested  in 
participating,  staffing  constraints  necessitated,  at  least  initially,  a  very 
limited  involvement;  however,  there  was  a  definite  commitment  to  actively 
support  the  effort.  In  contrast,  the  program  manager  for  the  contractor  was 
skeptical. 


Contractor 

Program 

Manager’s 

Qualified 

Commitment 


The  contractor’s  program  manager  was  focused  on  maintaining  progress 
toward  success.  His  concern  regarding  the  collaboration  centered  on  the 
possibility  that  this  effort  would  burden  the  program  with  extra  work  and 
responsibilities  that  would  add  minimal,  if  not  negative,  value.  He  wanted 
added  value  that  would  contribute  to  the  success  of  the  program,  not  simply 
more  work. 
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Conditional  Commitment  was  only  partially  achieved.  Approval  was  given  to  proceed, 

Approval  with  the  provision  that  the  government  or  the  contractor’ s  program  manager 

Received  could  terminate  the  process  at  any  time  if  it  was  felt  that  no  value  added  was 

being  realized. 
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Section  2  Introductory  Presentations 


Description  The  SEI  conducted  introductory  presentations  on  team  risk  management  for 
both  government  and  contractor  personnel. 


Intent  of  the  The  intent  of  the  presentations  was  to  expose  personnel  throughout  the  pilot 

Presentations  project  and  the  organization  to  the  team  risk  management  approach. 


Content  of  the  The  presentations  addressed  all  aspects  of  the  team  risk  management  ap- 

Presentations  proach  and  details  on  the  collaboration. 


Contractor  The  contractor  presentations  were  conducted  at  the  contractor’s  facilities 
Presentations  and  involved  key  personnel  from  the  pilot  program,  personnel  from  other 
programs,  and  corporate-wide  support  personnel. 


Government  Presentations  were  also  made  to  government  personnel  at  government  facil- 

Office  ities.  Personnel  from  the  pilot  program  and  the  systems  engineering  (S/E) 

Presentations  staff  participated. 


Observations  In  general,  the  presentations  at  both  organizations  were  well  received,  but  a 
substantial  portion  of  the  personnel  in  attendance  expressed  some  skepti¬ 
cism.  Generally,  these  comments  reflected  concern  over  the  depth  of  the 
commitment  of  their  organization  to  the  effort.  For  example,  would,  in  fact, 
the  team  risk  management  process  be  supported  over  the  long  haul?  Would 
it  be  dropped  after  a  short  period  of  time?  Is  this  another  false  start  on  im¬ 
provement  that  gets  nowhere? 
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Section  3  Initial  Training 


Initial  The  initial  training  activities  involved  training  the  contractor’s  transition 

Training  team  members  in  the  baseline  processes  and  the  initial  planning  steps  of 

team  risk  management. 


Contractor  The  training  of  the  contractor’ s  transition  team  involved  a  four-hour  session 

Transition  where  the  details  of  the  baseline  activity  for  identifying  risks  and  the  con- 

Team  tinuous  processes  were  presented. 

Training 


Roles  of  the  The  training  centered  on  the  facilitation  that  the  contractor’s  transition  team 
Contractor  members  would  be  providing  throughout  the  process  and  their  immediate 
Team  roles  in  the  baseline  activities. 

Members 


Team^  The  plan  was  for  the  same  individuals  from  both  organizations  to  participate 

Building  throughout  the  entire  collaboration.  One  of  the  goals  of  this  initial  training 

was  to  establish  the  foundation  for  the  long-term  working  relationships 
among  the  members  of  the  contractor-SEI  transition  team. 


Logistics  and  While  much  of  the  focus  of  the  training  dealt  with  the  issues  relating  to  spe- 

Database  cific  processes  and  methods  of  team  risk  management,  other  aspects  such  as 

Issues  file  formats,  software  conventions,  and  information  transfer  methods  were 

discussed. 


Except  for  company-sensitive  data,  the  team  members  decided  to  use  email 
transfer  of  documents  and  information  as  the  routine  means  of  exchanging 
information  among  team  members.  This  communication  mode  greatly  facil¬ 
itated  the  information  management  aspects  of  the  remote  collaboration. 


Email  and 
Routine 
Communica¬ 
tions 
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Section  4  Adapt  to  the  Project 


Commitment  A  primary  consideration  in  the  transition  activities  was  a  commitment  by  the 
to  Adaptation  SEI,  in  cooperation  with  the  contractor’s  transition  team  members,  to  adapt 
the  methods  to  meet  the  needs  and  conventions  of  the  pilot  project  and  the 
organization. 

Continuous  While  initiated  early  in  the  process,  training  continued  throughout  the  col¬ 
and  laboration.  The  nature  of  the  training  became  more  of  a  collaborative  effort. 

Collaborative  with  the  SEI,  contractor,  and  project  team  members  cooperatively  establish- 
Training  ing  detailed  practices  that  best  fit  the  organization  and  were  consistent  with 

the  principles  of  team  risk  management. 


Value  of  The  knowledge  and  experience  of  the  contractor  personnel  in  software  pro- 

Contractor  cess  improvement  and  organizational  issues  were  extremely  valuable  in  de- 
Participation  fining  specific  practices. 

For  example,  the  spreadsheet  data  formats  were  incorporated  for  routine 
meeting  presentations  as  a  result  of  a  suggestion  by  the  software  manager 
on  the  project. 


Solicitation  Throughout  the  collaboration,  the  SEI  transition  team  regularly  probed 

of  Feedback  project  personnel  seeking  to  ascertain  their  perception  of  the  utility  and  ef¬ 

fectiveness  of  the  methods.  It  was  hoped  that  through  regular  feedback  from 
project  personnel,  and  prompt  response  to  that  feedback  by  the  transition 
team,  project  personnel  would  develop  a  greater  sense  of  involvement  and 
ownership  of  the  process.  As  a  result,  we  hoped  that  there  would  develop  a 
stronger  commitment  to  the  successful  implementation  of  team  risk  man¬ 
agement. 
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Section  5  Project  Risk  Baseline 


Initiating 

Activity 


Baseline  risk  identification  and  analysis  is  the  first  opportunity  to  introduce 
a  larger  number  of  the  project  staff  to  the  details  of  the  entire  process. 


Objectives  of 
the  Baseline 


The  risk  baseline  activities  served  three  key  purposes: 

•  to  continue  the  buy-in  process  and  to  extend  the  commitment  to  include 
personnel  at  all  levels  and  in  all  functional  groups  of  the  project 

•  to  develop  a  baseline  set  of  risks  within  each  organization 

•  to  lay  the  foundation  for  the  ongoing  (continuous)  process  steps 


Baseline 

Activities 


The  baseline  activities  were  conducted  over  a  three-day  period  and  consist¬ 
ed  of  [Higuera  93] 

•  introductory  presentation  to  project  participants 

•  four  group  interviews 

•  management  risk  evaluations 

•  results  briefing 


Introductory 
Presentation 
to  All  Project 
Participants 


The  introductory  presentation  consisted  of  a  45-minute  briefing  to  all  of  the 
project  personnel  who  would  be  participating  in  the  baseline  activities.  The 
objectives,  specific  activities,  and  general  nature  of  the  process  were  pre¬ 
sented.  A  key  element  in  this  presentation  was  the  continuing  focus  on 
achieving  the  buy-in  of  project  personnel.  It  was  hoped  that  this  event  would 
result  in  at  least  open-mindedness,  if  not  enthusiasm. 


Anxiety  and 
Overcoming 
the  Audit 
Mentality 


One  of  the  more  difficult  aspects  of  building  commitment  to  the  process  in 
project  personnel  is  overcoming  the  anxiety  that  arises  from  the  view  that 
this  is  another  “audit”  activity.  There  was  a  sarcastic  sense  evident  in  infor¬ 
mal  comments  that  another  organization  is  “here  to  help”  and  make  judg¬ 
ments  on  the  project. 


Becoming 
Part  of 
Routine 
Practices 


The  introductory  presentation  apprised  project  personnel  of  the  fact  that 
team  risk  management  is  not  an  external  evaluation  process,  but  rather  is  a 
process  that  will  become  part  of  the  routine  project  activities  and  is  inher¬ 
ently  nonjudgmental.  The  external  involvement  -  the  SEI  presence  -  is  nec¬ 
essary  to  facilitate  the  transition  into  the  organization,  and  the  collaboration 
is  intended  to  create  a  “risk  aware”  culture  within  their  organization  [SEI 
92],  [Kirkpatrick  92]. 
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Skepticism  It  was  observed  that  following  the  introductory  presentation,  most  of  the 

Persisted  project  staff  were  very  skeptical  and  were  awaiting  events  with  uncertainty. 

This  uncertainty  was  also  evident  at  the  start  of  each  of  the  group  interview 
sessions. 


Group  The  group  interview  [Carr  93]  is  the  primary  identification  method  used 

Interviews  during  the  baseline  identification  process  and  involves  questions  addressed 
to  peer  groupings  of  project  personnel  (groups  of  three  to  five  individuals). 
The  atmosphere  of  the  interviews  is  nonjudgmental,  nonattributional,  and 
confidential. 

Four  group  interview  sessions  were  conducted  for  the  baseline. 


Initial  At  the  outset  of  the  interviews,  a  level  of  uncertainty  and  skepticism  existed 

Anxiety  regarding  the  effectiveness  and  likelihood  of  success  for  the  endeavor.  As 

each  of  the  interview  sessions  progressed,  especially  following  the  identifi¬ 
cation  and  recording  of  the  initial  risk  for  the  group,  a  greater  level  of  in¬ 
volvement  was  evidenced. 


Observations  In  general,  the  initial  anxiety  of  project  personnel  was  replaced  by  a  more 
and  Lessons  open  and  involved  participation.  This  evolution  was  evidenced  in  the  eval- 
Learned  uation  comments  made  by  the  participants.  A  representative  comment  re¬ 

garding  the  interview  process  was  that  it  was  a  “comfortable  setting  to  ex¬ 
press  concerns.” 


Management 

Evaluation 

Sessions 


As  part  of  the  baseline,  there  are  management  sessions  where  the  most 
important  risks  are  identified  and  prioritized.  These  risks  are  selected  from 
among  all  of  the  risks  identified  during  the  group  interviews  [Higuera  93], 
and  the  Continuous  Risk  Management  Guidebook^ .  These  sessions  are  the 
mechanism  to  focus  on  the  most  important  risks  quickly  and  place  them  in 
a  management-defined  priority  order. 


1.  Continuous  Risk  Management  Guidebook.  Dorofee,  Audrey;  Walker,  Julie;  Alberts,  Christopher; 
Higuera,  Ronald;  Murphy,  Richard;  Williams,  Ray.  (to  be  published  June  1996).  Pittsburgh,  PA;  Soft¬ 
ware  Engineering  Institute,  Carnegie  Mellon  University. 
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Nominal 
Group  and 
Comparison 
Risk  Ranking 


The  management  evaluations  include  an  implementation  of  the  Nominal 
Group  Technique  [Scholtes  88],  comparison  risk  ranking  [FitzGerald  90] 
and  consensus-based  decision-making  processes.  These  techniques  enable 
management  to  identify  the  most  important  risks  to  the  project  and  begin  to 
establish  a  common  management  understanding  of  each  of  the  most  impor¬ 
tant  project  risks.  This  management  comparison  risk-ranking  activity  is  fun¬ 
damentally  a  consensus-based  process  which  arranges  into  priority  order  the 
most  important  risks,  as  identified  in  the  management  review  session. 


Consensus 
and  the 
Program 
Manager 


While  consensus  is  sought  within  the  management  sessions,  the  consensus- 
based  decision-making  processes  are  defined  such  that  the  program  manag¬ 
er  maintains  the  authority  and  the  responsibility  for  the  outcome;  and  ulti¬ 
mately,  the  results  of  the  process  belong  to  the  program  manager.  The  con¬ 
sensus-based  processes  are  effective  methods  to  foster  a  shared  vision,  a 
systems  perspective,  open  communications,  and  the  formulation  of  proac¬ 
tive  strategies  among  management  personnel. 


Results  The  final  step  in  the  baseline  risk  assessment  is  a  presentation  of  the  results. 

Briefing  without  attribution  to  any  group  or  individual.  This  results  briefing  is 

conducted  as  a  formal  presentation  to  all  project  personnel  who  participated 
in  the  assessment  process.  While  this  step  is  the  conclusion  of  the  baseline 
risk  assessment,  this  presentation  is  also  the  forum  to  initiate  the  continuous 
processes  of  team  risk  management. 


Enthusiasm 


Following  the  baseline,  there  was  a  mix  of  skepticism  and  enthusiasm.  The 
baseline  seemed  to  stimulate  interest  and  action;  in  general  there  was  eager¬ 
ness  to  get  started. 
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Section  6  Install 


Installation  The  installation  process  involved  establishing  the  infrastructure  within  the 
Process  project  to  continuously  manage  the  risks  identified  in  the  baseline  and  to 

identify  newly  emerging  risks. 


Summary  of 

Routine 

Activities 


The  basic  elements  of  the  installation  process  are 

•  establish  risk  tracking 

•  start  routine  risk  identification  and  analysis  (RRIA) 

•  plan  first  risks 

These  were  addressed  by  establishing  a  number  of  routine  activities  to  man¬ 
age  risks  within  the  project.  These  activities  were  the  following: 

•  A  risk  management  board  was  established  and  regular  meetings  were 
held,  approximately  monthly. 

•  Risks  were  discussed  at  each  weekly  project  meeting. 

•  Risks  and  their  status  were  included  as  an  agenda  item  at  the  monthly 
project  meetings 


Risk 

Management 

Board 


The  risk  management  board  consisted  of  the  project’s  software  manager, 
lead  systems  engineer,  and  program  manager.  Occasionally,  key  technical 
personnel  were  included  to  address  specific  risks. 


Meetings  of  the  board  were  often  conducted  just  prior  to  coordination  meet¬ 
ings  held  between  the  contractor  and  the  SEI. 


Weekly 

Meeting 

Discussions 


The  status  of  risks  and  mitigation  plans  were  discussed,  and,  as  needed,  new 
risks  were  presented  as  part  of  the  regular  weekly  project  meetings  held  by 
the  project. 


Change  of  While  the  routine  activities  were  initially  viewed  as  “add-on”  work,  this  per- 

Perspective  ception  gradually  changed  and  risk  management  activities  were  seen  as  in¬ 

tegral  to  meetings  and  ongoing  project  management  activities.  In  fact,  as  the 
process  matured  and  the  project  management  staff  became  more  comfort¬ 
able  with  and  confident  in  risk  management,  the  risk  management  board  be¬ 
came  less  a  distinct  entity,  and  the  risks  were  routinely  managed  within  the 
context  of  the  regular  project  management  meetings. 
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Observation  observed,  “Everyone  is  involved  through  the  weekly  meetings.” 
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Section  7  Establish  Risk  Tracking 


Description  This  activity  establishes  tracking  efforts  to  provide  managers  with  the  status 
of  the  baseline  risks  as  plans  are  developed  and  put  into  place. 


Initial  Effort  The  initial  tracking  mechanism  was  established  by  the  software  manager 
Involved  a  and  involved  a  spreadsheet  listing  each  the  top  N  risks  identified  in  the  base- 

Spreadsheet  line. 


Extended  Use  Later  in  the  collaboration,  the  spreadsheet  was  extended  to  include  joint 
risks  (risks  that  are  jointly  shared  between  the  government  and  contractor) 
and  formed  the  key  instrument  for  reviews  conducted  by  the  risk  manage¬ 
ment  board. 


ModiHed  and  As  the  installation  of  the  team  risk  management  processes  progressed,  a  for- 
Generated  mal  risk  database  (which  included  a  broad  range  of  risk  data,  including  mit- 

from  the  igation  strategies  and  plans)  was  established  for  the  project. 

Database 

The  spreadsheet  and  the  database  were  maintained  separately  but  were  kept 
synchronized.  The  spreadsheet  continued  to  be  the  key  instrument  for  track¬ 
ing  the  most  important  risks  in  the  project. 


Quality  Initially,  the  spreadsheet  was  maintained  by  the  project’s  software  manager 

Assurance  while  the  database  was  maintained  by  the  contractor’s  transition  team  mem- 

Participation  bers.  Later  in  the  project,  the  spreadsheet  was  generated  by  members  of  the 

contractor’s  transition  team;  ultimately  these  responsibilities  were  trans¬ 
ferred  to  the  quality  assurance  group  within  the  contractor  organization. 


Gradual  Throughout  the  collaboration,  there  was  a  gradual  evolution  of  the  spread- 

Evolution  sheet  contents  and  use.  In  addition,  as  part  of  the  tracking  and  overall  risk 

management  process,  the  quality  assurance  group  began  to  participate  di¬ 
rectly  in  the  risk  management  activities.  This  involvement  included  partici¬ 
pation  in  risk  reviews  and  data  management  support. 


Lessons  Having  someone  assigned  to  be  responsible  for  reporting  progress  and  sta- 

Learned  tus  was  necessary  to  “inspire”  effort. 

Without  a  due  date,  it  was  difficult  to  get  action  taken. 
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Section  8  Start  RRIA 


Description  Following  the  project  risk  baseline,  a  voluntary  method  for  continuous  risk 
identification  and  analysis  was  established  to  identify  and  analyze  new  risks 
as  they  emerged  in  the  project. 


Communica-  The  existence  of  and  procedures  for  the  new  risk  identification  process  were 
tion  communicated  in  a  memorandum  to  all  project  staff.  In  this  memorandum 

all  project  personnel  were  encouraged  to  identify  new  risks  and  the  anony¬ 
mous  character  of  the  process  was  noted. 


Anonymous  The  initial  method  established  for  new  risk  identification  was  an  anonymous 

Submission  submission  procedure,  a  drop  box  located  in  the  organization’s  library,  for 

risk  identification. 


Risk  Form  New  risk  submission  forms  were  available  at  the  drop  box  location  and  dis¬ 
tributed  with  the  memorandum  announcing  the  process. 


Ineffective  The  voluntary  submission  process  did  not  work,  and  no  new  risks  were  iden¬ 
tified  through  this  process. 


Routine  Risk  As  part  of  the  risk  issues  agenda  item,  new  risk  identification  was  integrated 
Identification  into  the  regular  weekly  project  meetings  and  into  the  risk  management 
board  reviews.  For  this  process  the  risk  spreadsheet  was  used  as  a  cue  for 
risk  identification. 


Electronic  There  were  discussions  about  implementing  the  voluntary  submission  as 

Submission  part  of  the  existing  quality  assurance  problem-reporting  system.  However, 
this  was  never  implemented. 


Lessons  The  initial  attempt,  which  focused  on  anonymity  and  voluntary  submission. 

Learned  did  not  work,  but  enabled  project  and  contractor  personnel  to  participate 

with  the  SEI  to  define  methods  that  worked  for  the  project. 
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Section  9  Plan  First  Risks 

Initial  The  plans  and  actions  for  addressing  risks  were  initiated  immediately  fol- 

Planning  lowing  the  baseline  activities.  Most  of  the  discussions  regarding  risks  were 

conducted  in  risk  management  board  meetings  and  as  part  of  the  routine 
project  meetings. 


Top  N  Most  of  the  initial  focus  was  on  the  top  N  baseline  risks  for  the  project.  Ac¬ 

tion  items  and  responsibilities  were  assigned  for  these.  Subsequent  planning 
efforts  then  centered  on  the  more  complex  of  these  risks. 


Action  Items  Planning  initially  consisted  of  the  determination  and  assignment  of  action 
items  to  individuals. 


Lessons  It  is  not  enough  to  take  an  action.  There  needs  to  be  a  means  to  ensure  that 

Learned  the  action  was  effective. 

Action  items  are  insufficient  for  complex  risks;  formal  action  plans  and 
group  action  planning  sessions  are  required. 
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Section  10  Build  Government  and  Contractor 
Team  and  First  Team  Review 


Team  After  baseline  risks  were  identified  and  analyzed  for  both  the  government 

Reviews  and  the  contractor,  the  first  formal  joint  event,  the  team  review,  was 

conducted. 


Team  Review  The  team  review  is  a  joint  meeting  of  the  government  and  contractor  pro¬ 
gram  managers  and  their  immediate  staff  to  discuss  and  prioritize  risks.  This 
session 


Original 

Process 

Modified 


•  brings  together  each  program  manager’s  list  of  current  top  risks 

•  enables  a  joint  review  of  the  status  of  the  key  risks  and  their  mitigation 
plans 

•  maintains  continuity  between  new  risks  and  the  results  of  the  previous 
team  review 

•  helps  assure  a  common  understanding  of  the  most  important  risks  to  the 
program 

•  assigns  responsibilities  for  the  risks 

•  builds  and  maintain  momentum  in  government-contractor  team  risk 
management 

Comparison  risk  ranking  (CRR)  was  originally  intended  as  the  method  for 
prioritizing  the  risks  at  the  team  review.  At  the  request  of  contractor  man¬ 
agement  personnel  the  multi-voting  method  was  used  instead  of  the  CRR. 
The  concern  was  that  the  CRR  would  involve  “too  much”  time  (see  Contin¬ 
uous  Risk  Management  Guidebook^). 


Format  of  the  The  initial  team  review  was  a  face-to-face  meeting  of  all  parties;  later,  most 

Reviews  of  the  team  reviews  were  conducted  as  teleconferences,  involving  three 

separate  geographical  locations. 

Initially,  team  reviews  were  facilitated  by  the  SEI;  later  reviews  were 
facilitated  by  members  of  both  the  contractor  and  government  transition 
teams. 


A  ^  The  contractor  expressed  a  reservation  about  the  initial  series  of  team 

Reservation  reviews:  “early  reluctance  with  [the]  team  to  put  everything  on  the  table.” 
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Positive  The  first  team  review  was  well  received  as  evident  in  the  evaluation 

Response  comments; 

•  fostered  a  team  rationale  and  relationship 

•  outstanding  team-building 

•  better  understanding  of  the  other  side 

•  [customer]  was  included  in  sessions  and  knew  when  things  were  coming 


Lesson  While  a  certain  amount  of  “finger  pointing”  occurred  at  one  point,  when  it 

Learned  became  obvious  that  the  contractor  could  assign  risks  to  the  government, 

more  confidence  in  the  process  was  evidenced.  The  building  of  trust  and 
confidence  was  a  gradual  process. 


Conclusions  The  first  few  team  reviews  should  be  face-to-face  to  establish  trust  and  build 
a  team  (cooperative  atmosphere).  After  these  initial  events,  teleconferenc¬ 
ing  or  other  remote  communication  methods  can  provide  an  effective  and 
more  cost-effective  mode  of  interaction. 


1.  Continuous  Risk  Management  Guidebook.  Dorofee,  Audrey;  Walker,  Julie;  Alberts,  Christopher; 
Higuera,  Ronald;  Murphy,  Richard;  Williams,  Ray.  (to  be  published  June  1996).  Pittsburgh,  PA;  Soft¬ 
ware  Engineering  Institute,  Carnegie  Mellon  University. 
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Section  11  Joint  Action  Planning 


Joint  Action  Joint  action  planning  sessions  are  meetings  held  between  the  customer  and 
Planning  supplier  to  generate  plans  for  specific  joint  risks. 


Joint 

Participation 


Key  management  and  appropriate  technical  personnel  from  both  the  gov¬ 
ernment  and  contractor  participated  in  these  sessions. 


Format  of  the  The  process  steps  for  a  joint  planning  session  are  the  following: 

Session  ,  Preselect  a  risk(s)  for  the  session. 

•  Use  cause-and-effect  diagramming  to  identify  and  prioritize  root  causes. 

•  Identify  the  criteria  for  use  in  comparing  strategies. 

•  Use  brainstorming  to  generate  alternative  strategies  to  address  root 
causes,  and  apply  list  reduction  methods  to  identify  the  “vital  few.” 

•  Generate  the  basis  for  a  task  plan  (actions  and  success  criteria), 
preliminary  schedule,  and  resources  and  reporting  mechanisms. 


Lessons  One  of  the  important  lessons  learned  was  that  while  routine  team  reviews 

Learned  could  be  conducted  using  teleconferencing  technologies,  joint  action 

planning  needs  to  be  face  to  face.  (Perhaps  videoconferencing  technologies 
will  enable  these  to  be  held  remotely?) 

Although  good  facilitation  was  required,  extensive  explanation  of  the 
process  before  beginning  the  session  was  not  required. 

A  formal  task  plan  was  not  necessary.  Minutes  of  the  session  and  an  action 
item  list  with  assignments  and  due  dates  was  sufficient  to  monitor  progress. 
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Section  12  Establish  Continuous  Processes 

Continuous  The  team  risk  management  processes  of  identification,  analysis,  action 
Processes  planning,  tracking,  and  control  are  continuous  processes. 


Development-  The  details  of  the  action  planning  process  were  not  finalized  and  were  being 
al  Planning  developed  as  part  of  the  collaboration.  These  initial  efforts  provided  a  basis 

Process  for  developing  key  principles  for  action  planning. 


Planning  A  group  planning  session  was  held  to  generate  a  plan  for  one  of  the  more 

Session  pressing  risks  faced  by  the  contractor.  The  planning  session  involved  the 

project  software  manager  and  two  key  technical  personnel. 


Facilitated  by  The  planning  session  was  facilitated  by  the  transition  team,  with  SEI  per- 
the  SEI  sonnel  providing  the  leadership  in  the  process.  The  basic  methods  employed 

included  brainstorming,  cause-and-effect  analysis,  and  consensus-building 
and  voting  activities.  The  basic  process  followed  that  of  the  joint  action 
planning  event  (refer  to  Chapter  4,  Section  13). 


Feedback  and  comments  were  solicited  on  the  effectiveness  of  the  planning 
session.  There  was  a  feeling  that  it  was  a  “meaningful  exercise”  that  drew 
on  requisite  expertise  to  help  with  the  process.  Specific  lessons  learned  are 
summarized  below: 

•  The  cause-and-effect  diagram  used  during  the  analysis  process  served 
primarily  as  a  mechanism  for  achieving  a  common  understanding. 

•  The  criteria  for  evaluating  the  risk  mitigation  strategy,  while  established 
early  and  used  to  constrain  the  strategy  generation,  evolved  during  the 
process. 

•  Because  of  the  time  and  resources  involved,  it  was  felt  that  group  planning 
sessions  would  be  effective  primarily  for  the  more  difficult  risks,  rather 
than  for  all  of  the  risks  being  managed. 

Coordination  To  facilitate  the  installation  and  support  the  continuous  execution  of  these 
Meetings  processes,  regular  coordination  meetings  were  held  at  the  client’s  facilities. 

These  meetings  enabled  regular  exchanges  between  SEI  team  risk 
management  personnel,  the  contractor's  program  management  staff,  and  the 
contractor's  transition  agents. 


Lessons 

Learned 


32 


CMU/SEI-95-TR-016 


Chapter  4  Implementation 

Section  12  Establish  Continuous  Processes 


Components 

of 

Coordination 

Meetings 


First 

Coordination 

Meeting 


Working 
Session  of  the 
Transition 
Team 


Working 
Session  with 
Project 


Meeting  with 
the  Program 
Manager 


Effect  of 

Coordination 

Meetings 


The  coordination  meetings  extended  between  a  half  to  a  full  day  and  in¬ 
volved  three  distinct  components  summarized  below: 

•  working  sessions  of  the  transition  team 

•  working  session  with  project  personnel 

•  meeting  with  the  program  manager 


The  first  coordination  meeting  was  held  at  the  client’s  facility  within  two 
weeks  of  the  completion  of  the  baseline.  Subsequent  meetings  were  held  ap¬ 
proximately  monthly  thereafter. 


Two  working  sessions  of  the  transition  team  were  held,  one  as  the  first  meet¬ 
ing  in  the  morning  and  one  as  the  final  meeting  before  departure. 

These  sessions  address  issues  relating  to  transition,  effectiveness  of  the 
methods,  introduction  of  changes  or  enhancements,  risk  database 
management,  and  general  organizational  as  well  as  program-specific  issues. 


A  working  session  with  project  personnel  participating  in  risk  activities  was 
held  as  the  second  meeting  of  the  day. 

In  this  session,  we  discussed  and  reviewed  the  risks,  plans  implemented, 
actions,  and  the  status  of  the  risks  and  associated  mitigation  plans.  Issues 
relating  to  the  effectiveness  of  the  processes,  recommendations  for 
improvements,  and  issues  relating  to  the  transition  of  the  processes  into  the 
organization  were  addressed.  This  meeting  was  the  forum  for  face-to-face 
communication,  affecting  change,  stimulating  continued  activity  on 
program  risks,  and  making  decisions  on  actions. 


In  this  meeting,  the  program  manager  is  apprised  of  the  results  of  the  day's 
activities  and  the  status  of  the  transition  process,  and  his/her  input  is 
solicited  on  the  team  risk  management  process  as  well  as  the  transition 
effort. 


Generally,  the  coordination  meetings  proved  to  be  an  important  stimulus  to 
the  program's  risk  management  efforts  and  enabled  the  timely  addressing  of 
issues  that  emerged.  In  addition,  these  regular  meetings  facilitated  the  team¬ 
building  process  between  SEI  personnel,  program  management,  and  transi¬ 
tion  agents. 

For  example,  the  initial  approach  for  the  routine  risk  identification  and  anal¬ 
ysis  was  not  effective  in  identifying  new  risks.  This  issue  was  addressed  at 
a  coordination  meeting,  and  an  alternative  method  was  devised  and  success¬ 
fully  implemented. 
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Critical 

Feedback 


A  major  element  of  the  coordination  meetings  was  the  dialogue  that  oc¬ 
curred  between  project  personnel  and  the  transition  team  members,  and 
among  the  contractor  and  SEI  transition  team  members.  The  results  of  these 
dialogues  formed  the  basis  for  modifications  to  original  approaches.  Gener¬ 
ally  these  modifications  addressed  specific  program  and  contractor  require¬ 
ments  and  were  aimed  at  improving  overall  effectiveness. 
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Section  13  Repeat  and  Improve 


Ongoing  Improvement  and  continuation  of  the  processes  were  facilitated  by  the  co- 

Process  ordination  meetings.  These  meetings  provided  encouragement  and  stimulus 

to  the  project  and  provided  the  forum  for  project  members  to  comment  on 
and  initiate  changes  in  the  processes.  This  was  viewed  as  important,  espe¬ 
cially  as  the  project  evolved. 


Tele¬ 

conferences 


After  the  first  team  review  was  completed,  risks  were  included  as  part  of  the 
discussions  at  weekly  teleconference  calls  conducted  between  the  contrac¬ 
tor  and  government. 


Observations 

on 

Commitment 

and 

Improvement 


The  success  of  this  collaboration  relied  heavily  on  the  commitment  and 
leadership  of  strong  supporters  in  key  positions  in  the  project.  At  about  nine 
months  into  the  collaboration,  a  key  supporter  (the  software  manager)  left 
the  project.  Fortunately,  the  individual  who  assumed  the  position  of  soft¬ 
ware  manager  on  the  project  also  believed  in  the  value  of  risk  management. 
His  continuing  support  was  important  in  ensuring  that  the  processes  were 
sustained  and  that  improvements  were  made  to  meet  the  changing  needs  of 
the  evolving  project. 


Broad 

Commitment 


In  addition  to  the  personal  level  of  commitment,  the  commitment  of  the 
larger  organization,  in  the  form  of  the  contractor’s  transition  team  members, 
is  important.  These  individuals  were  assigned  the  responsibility  and  ac¬ 
countability  to  ensure  that  risk  management  was  effective  within  the  project 
and  in  transition  into  the  larger  organization. 


Database 

Improve¬ 

ments 


Throughout  the  collaboration,  a  number  of  improvements  were  made  to  the 
database  and  templates  (e.g.,  risk  information  sheets)  that  were  used.  Most 
of  these  were  suggested  by  project  personnel  in  order  to  provide  additional 
information  or  clarify  the  presentation.  For  example,  the  spreadsheet  format 
was  modified  to  include  the  type  of  risk  and  joint  risk  identification  number. 


Encouraged  A  key  aspect  of  the  collaboration  approach  was  to  encourage  the  contractor 

Innovation  organization  to  modify  the  templates  to  meet  their  needs. 


Closure 


Early  on,  there  was  interest  in  establishing  explicit  closure  processes  for  the 
risks.  There  was  a  desire  to  show  progress  on  the  risks. 
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Lessons 
Learned  on 
Closure  of 
Risks 


“Hold  Risk” 

Lesson 

Learned 


The  desire  to  show  progress  resulted  in  premature  closures  of  risks.  In  some 
cases,  actions  were  completed  on  a  risk  and  the  risk  was  closed,  only  to  re- 
emerge  a  few  weeks  later.  As  the  more  formal  planning,  tracking,  and  con¬ 
trol  processes  were  established,  the  closure  of  the  top  risks  became  a  joint 
decision  between  both  the  government  and  contractor  program  managers. 


The  risks  that  were  neither  part  of  the  top  N  nor  included  in  the  joint  list  with 
the  government  were  identified  as  “hold  risks.”  A  process  was  defined  to 
regularly  review  this  list,  as  a  cue  for  new  risk  identification  or  to  note 
whether  conditions  had  changed,  perhaps  making  a  risk  in  this  category 
more  important  to  the  program.  This  review  was  to  be  conducted  by  the  risk 
management  board. 

Despite  SEI  encouragement,  the  encouragement  of  the  contractor  transition 
team  members,  and  the  general  agreement  among  project  personnel  that  the 
hold-risk  review  approach  could  provide  an  effective  method  for  risk  iden¬ 
tification,  the  method  was  never  formally  adopted,  although  one  member  of 
the  risk  management  board  occasionally  and  informally  reviewed  these 
risks.  As  a  result  of  not  reviewing  the  hold  risks  more  formally,  one  such 
risk  became  an  issue  which  required  subsequent  resolution. 
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Section  14  Closure  of  the  Collaboration 


Formal  As  the  last  formal  interaction  between  the  contractor  organization  and  the 

Closure  SEI,  a  closure  session  was  conducted.  Due  to  funding  constraints,  this  oc¬ 

curred  approximately  four  months  before  the  completion  of  the  project. 


Continued 

Through 

Project 

Completion 


With  the  disengagement  of  the  SEI  from  the  effort,  team  risk  management 
was  continued.  Risk  management  processes  were  facilitated  by  the  client 
members  of  the  transition  team,  with  risks  being  managed,  updated,  mitigat¬ 
ed,  and  closed. 


Closure  The  closure  of  the  collaboration  consisted  of  a  formal  interview  by  SEI  per- 

Interview  sonnel  of  the  key  project  personnel,  including  the  program  manager,  soft¬ 

ware  manager,  and  senior  system  engineer  on  the  project.  A  formal  ques¬ 
tionnaire  was  used  and  the  responses  were  recorded. 


Ground  Rules  The  ground  rules  of  the  interviews  were  that  comments  would  not  be  attrib¬ 
uted  to  anyone  in  the  group  but  would  be  used  in  generalized  form,  as  anec¬ 
dotal  evidence  in  reporting  the  results  of  the  collaboration. 


Lessons 
Learned: 
Team  Aspect 
and 

Communica¬ 

tion 


The  comments  and  lessons  learned  from  the  session  are  integrated  into  ap¬ 
propriate  sections  of  this  report.  The  interviews  proved  valuable  as  a  final 
commentary  on  the  entire  transition  effort,  and  provided  detailed  comments 
and  a  broad  perspective  on  the  collaboration.  A  key  theme  that  emerged  was 
a  strong  sense  of  teaming  that  the  participants  felt  characterized  the  effort, 
resulting  in  a  fundamental  cultural  change  of  better  communication  on  the 
project. 
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Section  15  Continuation 

FoIIow-on  As  part  of  the  collaboration,  the  SEI  participated  in  the  implementation  of 

Project  team  risk  management  within  another  project.  The  contractor’s  transition 

team  members  led  the  effort,  including  the  baseline  activities.  The  SEI  par¬ 
ticipated  as  part  of  the  baseline  team,  but  played  an  advisory  role  and  sup¬ 
ported  the  activities  as  session  recorder. 


Baseline  The  baseline  review  was  conducted  by  personnel  on  the  contractor’s  transi¬ 

tion  team.  One  member  of  the  technical  staff  from  the  SEI  participated  in 
the  baseline  review,  not  as  a  team  leader,  but  rather  as  a  working  member  of 
the  team. 


One  detailed  process  change  was  made  in  that  copies  of  the  SEI  risk  taxon¬ 
omy  [Carr  93]  were  distributed  prior  to  the  interviews.  This  proved  to  be 
disruptive  to  the  interview  process.  Participants  entered  the  interview  ses¬ 
sions  with  a  prepared  list  of  risks.  This  seemed  to  subdue,  at  least  at  the  out¬ 
set,  the  nominally  very  interactive  dialogue  of  the  interview;  individuals 
simply  read  their  risks,  rather  than  reacting  to  the  specific  interview  ques¬ 
tions. 


While  explicitly  not  recommended  by  the  interview  guidelines,  one  of  the 
interviewers  for  the  baseline  was  also  assigned  to  the  project.  This  was  ne¬ 
cessitated  due  to  resource  constraints  within  the  contractor  organization. 
During  the  interview  conducted  by  this  individual,  some  of  the  participants 
were  reticent  and  others  were  argumentative,  especially  in  discussions  of 
specific  areas  of  the  project  that  involved  responsibilities  that  were  shared 
by  the  interviewer. 


Application  for  The  success  of  the  team  risk  management  approach  on  the  pilot  project  has 
Other  Projects  provided  the  stimulus  for  the  contractor  to  include  risk  management  in  other 
proposals  and  projects. 


Project 

Involvement 


Change  in  the 

Baseline 

Process 
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Selected  This  chapter  presents  a  set  of  comments  made  by  personnel  from  within  the 

Comments  contractor  organization.  These  are  grouped  by  topic  or  key  issue. 

The  comments  presented  below  were  provided  by  contractor  personnel, 
both  project  and  nonproject  personnel,  and  indicate  a  gradual  evolution  of 
team  risk  management  into  a  routine  and  valued  component  of  the  project. 

Note:  Brackets  are  used  to  identify  editorial  additions  that  were  employed 
to  improve  the  clarity  of  the  quotes  and  comments. 


Skepticism 

Gradually 

Replaced 


One  of  the  gratifying  aspects  of  this  effort  was  the  fact  that  the  skepticism, 
very  evident  at  the  outset,  was  gradually  replaced  with  active  involvement 
in  the  process. 

•  [There]  was  a  fear  we’d  be  chasing  “ghosts,”  but  TRM  generated  “real 
issues”;  the  process  dealt  with  “ghosts”  and  let  us  focus  on  the  top  risks. 

•  There  was  some  fear  that  it  [TRM]  might  be  an  inhibitor,  “inhibit”  things. 

•  I  was  a  fence-sitter  when  all  this  began,  but  through  this  process  we  were 
forced  to  develop  risk  management  in  a  very  open,  democratic  style. 


Culture 
Change 
Comments 
by  Project 
Personnel 


The  collaboration  gradually  resulted  in  a  cultural  change  within  the  project. 

•  [There  was  a]  daily  change  in  focus,  especially  in  those  risks  in  ranks  like 
4  and  5  (lower  than  the  top  most). 

•  There  was  “a  lot  of  value  [in  TRM]  and  [it]  re-oriented  our  thinking.” 

•  [There  was  a]  Culture  change:  risks  now  in  the  forefront  at  weekly  and 
monthly  project  meetings  —  you  can  deal  with  them  now. 


Costs  While  the  explicit  cost  savings  was  difficult  to  quantify  in  this  effort,  some 

of  the  observations  presented  below  indicate  the  perception  of  value  added 
and  cost  avoidance  evidenced  in  the  project. 

•  [Team  risk  management]  helped  with  cost  avoidance.  Would  have  cost 
more  if  [we]  had  not  been  able  to  resolve  things  in  a  timely  manner. 

•  An  impression  of  the  team  risk  management  approach  is  reflected  in  the 
comment,  “Focus  is  on  cost-effective,  minimum  resource  utilization.” 
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Flexibility  in 
the 

Transition 


Support 

Tools 


General 

Observations 


Issues  and 
Criticisms 


The  flexibility  in  implementation  details  was  important  and  helped  the 
client  to  feel  part  of  the  process  and  a  sense  of  ownership. 

•  The  integrated-incremental  introduction  of  process  was  effective  in 
getting  the  methods  and  tools  accepted. 

•  Initially,  the  process  was  viewed  as  an  add-on.  Need  to  be  viewed  as  add¬ 
on  initially;  later  [it  was]  more  integrated,  less  add-on  sense. 


While  tools  for  supporting  team  risk  management  were  not  a  primary  issue, 

it  is  noteworthy  that  tools  were  seen  as  valuable. 

•  Want  more  tools  to  support  process:  online  as  opposed  to  paper;  cost  to 
get  started  with  databases,  etc. 

•  Use  of  spreadsheet  for  data  collection/reference  helped. 

•  More  “tools”  [to  support  the  process],  more  widely  available.  Also  PC 
tools  [to  handle  data]. 

•  If  we  are  going  to  practice  this  on  an  organizational  level  [need]  tools/ 
computer  tools  [data  management,  etc.,  and  a]  manual  needed. 


These  comments  provide  a  broad  assessment  of  the  value  of  the  team  risk 

management  process. 

•  Improved  communications  between  customer  and  supplier  was  most 
successful  aspect. 

•  Teaming  is  what  worked;  team  was  outstanding.  [It  was]  noncombative. 

•  [The  approach]  avoided  “contractual  nuances.” 

•  Having  a  risk  in  the  forefront  might  have  influenced  what  you  do,  e.g.,  you 
might  have  done  A-B-C  but  did  C-B-A  instead  because  of  the  identified 
risks. 

•  Started  as  a  software  process  and  quickly  became  system-oriented,  which 
was  the  right  thing  to  do. 

•  Getting  [risk]  in  front  of  people  on  a  regular  basis  makes  people  aware  of 
it. 

•  [Risk  management]  stimulated  more  timely  resolution  rather  than 
identification  of  problems. 

A  number  of  issues  and  criticisms  help  to  identify  aspects  of  the  transition 

and  collaboration  that  needed  improvement. 

•  Early  reluctance  with  team  to  put  everything  on  the  table. 

•  Mitigation  of  toughest  risks  was  the  hardest  thing  to  accomplish. 

•  Still  difficult  to  deal  with  long-term  risks;  more  comfortable  with  the  near- 
term,  concrete  things. 

•  [The  difference  between]  risk  and  problem  still  isn’t  always  clear,  even 
to  us. 
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Chapter  6  Observations  and  Lessons 

Learned 

Description  This  chapter  is  a  compendium  of  observations  made  by  the  transition  team 

and  a  summary  of  the  lessons  learned. 


Observations  A  number  of  important  observations  that  were  made  by  the  transition  team 
regarding  the  collaboration,  transition,  and  team  risk  management  are  sum¬ 
marized  below. 

•  Attaining  and  maintaining  effective  communications  is  the  central  issue  in 
the  government  -  contractor  relationship. 

•  The  collaboration  has  demonstrated  that  government  and  contractor 
perspectives  can  be  openly  shared. 

•  The  trust  between  all  partners  in  the  process,  including  the  SEI,  and 
confidence  in  the  team  risk  management  approach  itself  must  develop 
gradually  over  time. 

•  The  approach  successfully  provided  a  forum  for  communication  about 
what  are  often  perceived  as  negative,  unpleasant  issues.  This  success 
demonstrated  that  groups  can  work  effectively  together,  not  only  on  these 
issues  but  on  other,  less  sensitive  issues  and  build  and  sustain  cooperative 
relationships  throughout  the  program. 

•  The  adaptability  of  the  methods  was  valuable  and  facilitated  the 
integration  and  acceptance  of  the  team  risk  management  methods  into  the 
established  processes  of  the  organization. 

•  Incremental  adoption  of  the  processes  and  methods  was  effective. 

•  The  stimulus  of  regular  meetings  proved  invaluable  in  both  facilitating  the 
process  and  securing  feedback. 

Commitment  To  successfully  implement  this  technology,  it  is  vital  that  the  organization 
make  the  commitment  in  dollars  and  time,  and  assign  a  sufficient  number 
of  personnel  with  the  responsibility  and  concomitant  accountability  to  im¬ 
plement  the  technology  and  ensure  its  success. 
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Look  at  AH 
the  Risks 


Rigorous 

Closure 

Process 


Adaptable 

Implementa¬ 

tion 


Incremental 

Introduction 


Collaboration 

Was 

Important 


Need  to 
Provide  a 
Stimulus 


It  is  important  to  push  to  have  the  project  look  at  all  of  the  risks.  There  was 
a  reluctance  to  deal  with  any  more  than  the  most  important  risks.  Although 
a  process  was  developed  to  ensure  that  the  complete  list  was  reviewed  peri¬ 
odically  for  changes  (e.g.,  increased  probability  or  impact)  and  used  as  a 
stimulus  for  the  identifying  new  risks,  this  process  was  not  rigorously  fol¬ 
lowed. 

While  this  review  effort  should  be  conducted  as  a  low  level-of-effort  task,  it 
was  evident  that  this  review  can  identify  new  risks  and  enable  early  recog¬ 
nition  of  important  risks  that  require  management  attention. 


Throughout  the  pilot  effort,  risks  were  closed,  much  as  one  would  close  an 
action  item.  In  many  cases,  especially  early  in  the  process,  risks  were  closed 
prematurely.  It  is  important  to  adhere  to  a  rigorous  and  structured  closure 
process  to  help  prevent  premature  closures  of  risks.  For  example,  a  method 
used  in  the  team  review  required  agreement  of  both  program  managers  be¬ 
fore  closing  a  joint  risk. 


A  basic  approach  used  in  the  collaborative  development  involved  the  pre¬ 
sentation  of  proposed  methods  and  products,  and  their  subsequent  refine¬ 
ment  and  adaptation  based  on  the  client’s  needs  and  suggestions.  This 
adaptable  character  of  both  the  team  risk  management  approach  and  the  col¬ 
laboration  itself  appeared  to  be  a  key  factor  in  gaining  acceptance  of  risk 
management  concepts  in  general,  and  team  risk  management  in  particular; 
it  was  also  helpful  in  developing  confidence  in  the  methods  and  trust  across 
government,  contractor,  and  SEI  personnel. 


The  incremental  introduction  of  processes  proved  valuable  by  enabling 
project  personnel  first  to  master  parts  of  the  process,  make  modifications  as 
needed,  and  then  learn  additional  methods  gradually.  This  facilitated  the 
identification  of  improvements,  the  effective  use  of  the  methods,  and  the  in¬ 
tegration  of  the  approach  into  the  project. 


Substantial  value  was  derived  from  the  involvement  of  the  contractor’s 
project  and  nonproject  personnel.  Their  input  in  the  form  of  changes  to  the 
process  and  methods,  and  suggestions  for  the  redesign  of  tools  (forms, 
spreadsheets,  etc.)  helped  establish  more  of  a  sense  of  ownership.  The  pro¬ 
cesses  were  perceived  as  a  mutual  agreement,  rather  than  viewed  as  im¬ 
posed. 


One  of  the  key  lessons  learned,  which  arose  out  of  a  behavior  that  was  evi¬ 
dent  throughout  this  collaboration,  is  that  a  periodic  stimulus  is  required  to 
ensure  that  the  processes  are  executed. 
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Coordination 

Meetings 


Coordination 

Meetings 

Became 

Routine 


Evolved  to 
Become 
Routine  and 
Integral  to  the 
Project 

Near-Term 

Orientation 


It  was  observed  that  while  the  contribution  and  value  added  of  the  team  risk 
management  processes  were  acknowledged  early  in  the  collaboration  by 
project  personnel,  pressing  problems  and  project  demands  often  resulted  in 
risk  management  being  relegated  to  a  lower  priority  activity.  This  was  espe¬ 
cially  evident  during  the  first  few  months  of  the  collaboration. 

For  example,  as  circumstances  changed  or  new  issues  emerged,  these  were 
not  explicitly  documented,  and  occasionally  risk  discussions  were  terminat¬ 
ed  prematurely  or  not  held  at  all  due  to  “more  pressing  problems.”  Implicit 
actions  and  awareness  seemed  to  be  a  more  expeditious  way  to  deal  with 
these  risks. 

The  transition  team  found  that  the  regular  coordination  meetings  held  with 
the  project  proved  to  be  an  effective  stimulus  for  the  project.  These  meetings 
precipitated  action  and  prompted  the  project  to  deal  explicitly  with  risks  on 
an  ongoing  basis. 


In  the  later  stages  of  the  collaboration,  the  coordination  meetings  became 
increasingly  perfunctory  and  acted  as  a  focused  event  in  time  that  was  used 
to  convey  results  of  the  risk  management  efforts.  The  project  was  fully 
prepared  and  the  presentations  were  concise.  Issues  were  clearly  delineated 
and  the  time  required  to  review  the  risks  diminished  substantially. 

Similarly,  the  transition  team  coordination  meetings  become  increasingly 
routine,  dealing  less  with  the  specifics  of  the  project  and  addressing  more 
global,  follow-on  issues. 


As  the  collaboration  progressed,  the  practice  of  team  risk  management  be¬ 
came  a  routine  and  integral  part  of  the  program's  management  processes. 
The  discussions  of  risks  were  incorporated  into  regular  project  meetings, 
and  risk  management  was  seen  as  a  vital  part  of  the  project  management’s 
responsibilities. 


It  was  easy  for  the  project  to  delay  explicit  action  on  the  longer  term  (future) 
issues,  like  risk,  in  order  to  handle  the  immediate  crises.  This  may  be  due  to 
the  pressure  to  see  results  and  the  lack  of  immediacy  that  characterizes  risk. 
While  this  may  also  be  symptomatic  of  resistance  to  change,  a  concerted  ef¬ 
fort  must  be  undertaken  to  encourage  the  project  team  to  continue  to  be  vig¬ 
ilant  and  look  ahead. 

This  issue  continues  to  be  one  of  the  more  difficult  aspects  of  implementing 
effective  risk  management. 
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Chapter  7  Summary 


Key  Aspects  The  collaboration  was  successful  in  transitioning  team  risk  management 
into  a  pilot  project  and  fostering  its  adoption  in  other  projects  within  the 
larger  contractor  organization. 

The  aspects  of  this  effort  that  received  positive  feedback  and  provided  a 
basis  for  success  can  be  summarized  as  follows; 

•  Maintain  a  flexible  approach. 

•  Encourage  teamwork. 

•  Utilize  an  incremental  introduction. 

•  Integrate  into  other  program  management  processes. 

•  Provide  a  periodic  stimulus. 

•  Foster  a  champion. 


Flexible 


It  is  important  to  incorporate  flexibility  in  the  approach  in  order  to  enable 
modifications  to  improve  process  effectiveness  and  meet  the  unique  re¬ 
quirements  of  an  organization. 


Collaborative  The  contributions  of  contractor  personnel  both  within  and  outside  of  the 
involvement  project  proved  invaluable  in  defining  effective  enhancements  to  the  pro¬ 
cesses,  methods,  and  tools  of  team  risk  management.  As  the  collaboration 
evolved,  a  sense  of  teamwork  pervaded  all  aspects  of  the  effort. 


Incremental  An  incremental  approach  provided  the  format  for  gradual  introduction  of 
Introduction  additional  and  more  complex  activities,  and  enabled  modifications  of  pro¬ 
cesses  as  they  evolved. 


Integrate  into  The  concerted  effort  to  establish  risk  management  activities  as  a  normal  part 
Processes  of  the  project  and  integrate  these  into  established  project  management  activ¬ 

ities  fostered  acceptance.  As  a  result,  risk  management  was  viewed  as  an  in- 
tegral  part  of,  rather  than  an  add-on  to,  routine  project  activities. 


Stimulus  Is 
Required 


For  example,  regular  project  meetings  became  the  forum  for  conducting  re¬ 
views  of  risks  and  consolidating  risk  management  activities. 

A  stimulus  is  required  to  provide  encouragement,  to  act  as  a  prompt  to  ac¬ 
tion,  and  to  reinforce  the  successes.  The  regularly  scheduled  coordination 
meetings  proved  to  be  a  prompt  for  action  on  risks.  As  the  collaboration  ma¬ 
tured,  though,  the  need  for  the  stimulus  diminished  and  risk  management 
became  self  sustaining. 


CMU/SEI-95-TR-016 


45 


Chapter  7  Summary 


Champions  As  has  been  demonstrated  here  and  in  other  studies,  a  champion(s)  at  all  lev- 
Are  els  of  the  project  is  vitally  important  to  help  ensure  success.  For  this  effort 

Important  there  were  champions  of  the  approach  within  technical  and  multiple  man¬ 

agement  levels  of  both  the  government  and  contractor  organizations. 


Open 

Communica¬ 
tion  Can  Be 
Achieved 


While  the  installation  of  team  risk  management  practices  and  establishment 
of  trust  between  all  parties  was  a  gradual  process,  this  collaboration  has 
shown  that  government  and  contractor  perspectives  on  risks  can  be  openly 
shared.  These  successes  and  the  key  aspects  of  the  collaboration  that  are 
summarized  here  embody  a  central  issue  in  successful  transition:  attaining 
and  maintaining  effective  communications  among  all  parties  involved. 
Team  risk  management  and  the  collaborative  approach  described  here  pro¬ 
vide  a  forum  for  effective  communication  and  a  basis  for  cooperation  for 
successful  risk  management. 
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